Papers with behavioral probing
Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures (2026.findings-acl)
Copied to clipboard
| Challenge: | In-context learning is an emergent ability from pretrained Large Language Models (LLMs). |
| Approach: | They perform in-depth evaluations of in-context learning on transformers and hybrid large language models using behavioral probing and intervention-based methods. |
| Outcome: | The proposed model performs well on state-of-the-art transformer, state-space, and hybrid large language models. |
Probing the Category of Verbal Aspect in Transformer Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | a particular challenge is posed by ”alternative contexts” where either the perfective or the imperfective aspect is suitable grammatically and semantically. |
| Approach: | They investigate how pretrained language models encode the grammatical category of verbal aspect in Russian. |
| Outcome: | The proposed model has high predictive uncertainty about aspect in alternative contexts, the authors show . |
Rethinking Document-Level Relation Extraction: A Reality Check (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts push up performance boundaries of document-level relation extraction (DocRE) but these efforts are not promising. |
| Approach: | They construct four types of entity mention attacks to examine model robustness . they also have a close check on model usability in a more realistic setting . |
| Outcome: | The proposed model is based on a strong or untenable assumption in common . the model is robust under four types of mention attacks and usable in a realistic setting . |